1 Changelog

Main modifications with respect to previous versions:

2020-11-30

2020-11-06

2020-10-30

2020-09-24

2 Design and main assumptions

We simulate a two-arm trial with two binary endpoints according to the following parameters:

The correlation between the endpoints is assumed to be equal for both groups. We note that since the correlation is bounded between two values defined by (\(p_1^{(0)},p_2^{(0)},OR_1,OR_2\)), not all the combinations of (\(p_1^{(0)},p_2^{(0)},OR_1,OR_2,\rho\)) are possible. We calculate the correlation bounds for each combination of (\(p_1^{(0)},p_2^{(0)},OR_1,OR_2\)) and discard those scenarios in which the correlation is not within the valid range. The total number of scenarios is \(183\).

We consider the composite binary endpoint formed by endpoints 1 and 2. Based on the parameters (\(p_1^{(0)},p_2^{(0)},OR_1,OR_2,\rho\)), we compute:

The sample size used for the simulation is calculated to have \(0.80\) power to detect an effect of \(OR_1\) on the endpoint 1 at significance level \(\alpha = 0.05\).

2.1 Decision criterias

We assume that the endpoint 1 is more relevant for the clinical question than endpoint 2.

ARE Method:

We consider the Asymptotic Relative Efficiency (ARE) method to quantify the efficiency of using the composite endpoint over the endpoint 1 as primary endpoint of the trial. Based on the parameters (\(p_1^{(0)},p_2^{(0)},OR_1,OR_2,\rho\)), we compute:

  • ARE value;
  • ARE Decision: CE, if the composite endpoint is preferred; RE, if the endpoint 1 is preferred.

Ratio Sample sizes:

We calculate the sample size to have \(0.80\) power to detect an effect of \(OR_1\) on the endpoint 1, given the probability \(p_1^{(0)}\) under the control group, at significance level \(\alpha = 0.05\), say \(n_1\).

We also calculate the sample size to have \(0.80\) power to detect an effect of \(OR_*\) on the composite endpoint, given the probability \(p_*^{(0)}\) under the control group, at significance level \(\alpha = 0.05\), say \(n_*\). Note that for calculating the sample size for the composite endpoint have been used the values of the parameters (\(p_1^{(0)},p_2^{(0)},OR_1,OR_2,\rho\)).

The decision criteria is the ratio of the two sample sizes:

\[n_1 / n_*\]

so that if the ratio is smaller than \(1\), we use the relevant endpoint; whereas if the ratio is larger than \(1\), we use the composite endpoint.

2.2 Cases under study

We evaluate through simulations the statistical power and significance level under different situations and using different primary endpoints.

For each scenario, we compare the two groups based on three endpoints:

  • the relevant endpoint: we consider endpoint 1 as the primary endpoint;
  • the composite endpoint: we consider the composite endpoint formed by the union of endpoint 1 and endpoint 2 as the primary endpoint;
  • the selected endpoint: we select the primary endpoint at the end of the trial. We do so based on the ARE method and using blinded/unblinded data to estimate the probabilities under the control group and the correlation.

Remark: we simulate \(100000\) trials for each scenario and for each endpoint considered as primary endpoint;

For each scenario, we evaluate the following cases:

  1. Under alternative hypothesis (\(H_1\)):

    1. testing the group differences based on the endpoint 1.
    2. testing the group differences based on the composite endpoint.
    3. testing the group differences based on the selected endpoint.

  2. Under the null hypothesis (\(H_0\)):

    1. testing the group differences based on the endpoint 1.
    2. testing the group differences based on the composite endpoint.
    3. testing the group differences based on the selected endpoint.

When simulating under \(H_0\), we consider \(OR_1=OR_2=1\); whereas when simulating under \(H_1\), we consider the following situations:

  • Effect on both composite components: That is, we consider the previous mentioned values for \(OR_1\), \(OR_2\), and the resulting \(OR_*\).
  • Effect on only one component: That is, we plan the trial based on the previous mentioned values \(OR_1\), \(OR_2\), and the resulting \(OR_*\), but we simulate that \(OR_1=1\) or \(OR_2=1\).

2.3 Estimation correlation and ARE recalculation

We consider two different approaches for estimating the correlation between the endpoints 1 and 2.

Blinded approach:

Based on the observed responses in the pooled sample, we estimate the probabilities \(p_1\), \(p_2\), and \(p_*\), where \(p_k=\pi p_k^{(0)} + (1-\pi)p_k^{(1)}\) for \(k=1,2,*\) and \(\pi=n^{(0)}/n\). Assuming the expected effects for the endpoints 1 and 2 (say \(OR_1\) and \(OR_2\)) pre-specified in advance, we obtain estimates of the probabilities of each composite component under the control group \(p_1^{(0)},p_2^{(0)}\) and subsequently the estimates of the probabilities under the treatment group \(p_1^{(1)},p_2^{(1)}\).

Using the estimated probabilities for each composite component in each group (\(p_1^{(0)},p_2^{(0)},p_1^{(1)},p_2^{(1)}\)) and using the estimated pooled probability of the composite endpoint (\(p_*\)), we get an estimate of the correlation.

We calculate the ARE by using the estimated probabilities \(p_1^{(0)},p_2^{(0)}\) and the estimated correlation.

Unblinded approach:

Based on the observed responses in patients in the control group, we estimate the probabilities \(p_1^{(0)},p_2^{(0)},p_*^{(0)}\). Then, using these estimated probabilities, we obtain an estimate of the correlation between the Endpoints 1 and 2 in the control group \(\rho^{(0)}\).

We do the same for the treatment group, and then we obtain an estimate of the correlation in the treatment group \(\rho^{(1)}\).

We calculate the ARE by using the estimated probabilities \(p_1^{(0)},p_2^{(0)}\) and the mean of the two estimates of the correlation.

2.4 Correlation bounds

As mentioned before, Pearson correlation coefficient between two binary endpoints takes values between two bounds: denote by \(B_L(\cdot)\) the lower bound, and by \(B_U(\cdot)\) the upper bound. Both \(B_L(\cdot)\) and \(B_U(\cdot)\) are functions of the marginal probabilities of the binary endpoints.

Thus, in our context, the correlation between endpoints 1 and 2 in the control group lies within \(B_L(p_1^{(0)},p_2^{(0)})\) and \(B_U(p_1^{(0)},p_2^{(0)})\); whereas the correlation between them in the treatment group takes values between \(B_L(p_1^{(1)},p_2^{(1)})\) and \(B_U(p_1^{(1)},p_2^{(1)})\).

Since we assume that the correlation is the same in both treatment groups, we require that:

\[ max(B_L(p_1^{(0)},p_2^{(0)}),B_L(p_1^{(1)},p_2^{(1)})) \leq \rho \leq min(B_U(p_1^{(0)},p_2^{(0)}),B_U(p_1^{(1)},p_2^{(1)})) \]

In the estimation approaches explained above, we compute the correlation bounds based on the estimated probabilities and force the estimated correlation to take values within this interval. We notice that this interval can be different from the one calculated based on the true probabilities (those used to run the simulations).

3 Results

3.1 Under alternative hypothesis (\(H_1\))

3.1.1 Cases (1a) and (1b)

In what follows we compare the statistical powers obtained using the composite endpoint with the ones obtained using the endpoint 1 as primary endpoint. We also evaluate the concordance between the choice of the primary endpoint -according to the ARE value and the ratio of the sample sizes- and the difference in powers obtained in the simulation. In this case, we calculate the ARE at the beginning of the study, and then according to the parameters (\(p_1^{(0)},p_2^{(0)},OR_1,OR_2,\rho\)).

dataset$diff_powers = dataset$Test_Power_CE - dataset$Test_Power_RE

The following plot shows the relationship between the ARE value and the difference in power between using the composite endpoint (CE) or using the endpoint 1 (RE).

ggplot(dataset, aes(x=ARE, y=diff_powers, color=as.factor(decision))) + geom_point(size=2) + labs(y = "Difference in power (Power CE - Power RE)", x="ARE", color="Decision")

Cases in which there is no concordance between the decision and the power gain:

dataset$gain_power = ifelse(dataset$diff_powers>0, "CE", "RE")
dataset$gain_power = as.factor(dataset$gain_power)
dataset$decision = as.factor(dataset$decision)

summary(subset(dataset,dataset$gain_power != dataset$decision)[,c(3,4,18,19,20,21,22)])
##       OR1           OR2            OR_ce             ARE         decision
##  Min.   :0.6   Min.   :0.750   Min.   :0.6338   Min.   :0.8664   CE: 0   
##  1st Qu.:0.6   1st Qu.:0.750   1st Qu.:0.6669   1st Qu.:0.9118   RE:20   
##  Median :0.6   Median :0.750   Median :0.6787   Median :0.9351           
##  Mean   :0.6   Mean   :0.765   Mean   :0.6883   Mean   :0.9359           
##  3rd Qu.:0.6   3rd Qu.:0.800   3rd Qu.:0.7184   3rd Qu.:0.9657           
##  Max.   :0.6   Max.   :0.800   Max.   :0.7350   Max.   :0.9935           
##  samplesize_e1  samplesize_ce   
##  Min.   : 714   Min.   : 658.5  
##  1st Qu.: 714   1st Qu.: 700.5  
##  Median :1335   Median :1223.0  
##  Mean   :1149   Mean   :1091.4  
##  3rd Qu.:1335   3rd Qu.:1303.5  
##  Max.   :1335   Max.   :1317.5

3.1.2 Case (1c)

The following figure illustrates the empirical power we obtained using the composite endpoint (CE), the endpoint 1 (RE) or the selected endpoint (SE).

subd=dataset#subset(dataset,is.na(dataset$Test_Power_ES_ub)==F)

power_data <- data.frame(Power=c(dataset$Test_Power_ES,dataset$Test_Power_ES_ub,dataset$Test_Power_CE,dataset$Test_Power_RE),
                         Endpoint=c(rep("SE",length(dataset$Test_Power_ES)),
                                    rep("SE_ub",length(dataset$Test_Power_ES_ub)),
                                    rep("CE",length(dataset$Test_Power_CE)),
                                    rep("RE",length(dataset$Test_Power_RE)))
)

ggplot(power_data, aes(x=Endpoint, y=Power)) + geom_boxplot()

3.1.3 Comparing empirical powers - without interim analysis

For each combination of (\(p_1^{(0)},p_2^{(0)},OR_1,OR_2\)), we plot the empirical power for each design -Selected Endpoint with blinded estimation (SE), Selected Endpoint with unblinded estimation (SE_u), Composite Endpoint (CE), and Relevant Endpoint (RE)- according to the correlation \(\rho\). In the empirical powers for the Selected Endpoint, we plot the points in different colors depending on which endpoint would have been chosen using the ARE calculated from the true parameters (\(p_1^{(0)},p_2^{(0)},OR_1,OR_2,\rho\)).

For each combination of (\(p_1^{(0)},p_2^{(0)},OR_1,OR_2\)), we also present a table with:

  1. Corr: The values considered for the correlation in this scenario;
  2. ARE: The value of the ARE computed using the parameters used in the design, i.e., (\(p_1^{(0)},p_2^{(0)},OR_1,OR_2\)) and \(\rho\);
  3. %CE: The percentage of cases over the 100000 replications in which the selected endpoint was the composite endpoint using blinded data;
  4. %CE_u: The percentage of cases over the 100000 replications in which the selected endpoint was the composite endpoint using unblinded data.
p <- list() 
enum = 1 
it <- 1
for(i in 1:max(dataset$scenario)){
  sub=subset(dataset,dataset$scenario==i & dataset$p_init==1.0)
  # 
  p[[enum]] <-ggplot(sub, aes(x=corr, y=Test_Power_ES, color=as.factor(decision)))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset$p0_e1[it],",",dataset$p0_e2[it],",",dataset$OR1[it],",",dataset$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (ES)", x="Correlation", color="Decision") + coord_cartesian(ylim = c(0.60, 1))+ geom_path()+ theme(plot.title = element_text(size=9),legend.position = c(0.8, 0.2))
  p[[enum+1]] <-ggplot(sub, aes(x=corr, y=Test_Power_ES_ub, color=as.factor(decision)))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset$p0_e1[it],",",dataset$p0_e2[it],",",dataset$OR1[it],",",dataset$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (ES, unblinded)", x="Correlation", color="Decision") + coord_cartesian(ylim = c(0.60, 1))+ geom_path()+ theme(plot.title = element_text(size=9),legend.position = c(0.8, 0.2))
  # 
  p[[enum+2]] <-ggplot(sub, aes(x=corr, y=Test_Power_ES_SS, color=as.factor(ss_decision)))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset$p0_e1[it],",",dataset$p0_e2[it],",",dataset$OR1[it],",",dataset$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (ES)", x="Correlation", color="Decision SS") + coord_cartesian(ylim = c(0.60, 1))+ geom_path()+ theme(plot.title = element_text(size=9),legend.position = c(0.8, 0.2))
  p[[enum+3]] <-ggplot(sub, aes(x=corr, y=Test_Power_ES_ubSS, color=as.factor(ss_decision)))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset$p0_e1[it],",",dataset$p0_e2[it],",",dataset$OR1[it],",",dataset$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (ES, unblinded)", x="Correlation", color="Decision SS") + coord_cartesian(ylim = c(0.60, 1))+ geom_path()+ theme(plot.title = element_text(size=9),legend.position = c(0.8, 0.2))
  # 
  p[[enum+4]] <-ggplot(sub, aes(x=corr, y=Test_Power_CE))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset$p0_e1[it],",",dataset$p0_e2[it],",",dataset$OR1[it],",",dataset$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (CE)", x="Correlation", color="Decision") + coord_cartesian(ylim = c(0.60, 1))+ geom_path()+ theme(plot.title = element_text(size=9))
  p[[enum+5]] <-ggplot(sub, aes(x=corr, y=Test_Power_RE))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset$p0_e1[it],",",dataset$p0_e2[it],",",dataset$OR1[it],",",dataset$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (RE)", x="Correlation", color="Decision") + coord_cartesian(ylim = c(0.60, 1))+ geom_path()+ theme(plot.title = element_text(size=9))

  stable <- data.frame(Corr=sub$corr, 
                       ARE=round(sub$ARE,2),"%CE"=round(100*sub$decision_ES,2),
                       "%CE_u"=round(100*sub$decision_ES_ub,2),check.names=FALSE)  

  p[[enum+6]] <- ggtexttable(stable, rows = NULL, theme = ttheme(base_style = "default", base_size = 9))
  
  stable <- data.frame(Corr=sub$corr, 
                       SS=round(sub$ss_ratio,2),"%CE"=round(100*sub$decision_ES_SS,2),
                       "%CE_u"=round(100*sub$decision_ES_ubSS,2),check.names=FALSE)  

  p[[enum+7]] <- ggtexttable(stable, rows = NULL, theme = ttheme(base_style = "default", base_size = 9))

  enum=enum+8 
  it <- it + dim(subset(dataset,dataset$scenario==i))
}
marrangeGrob(p,ncol=4,nrow=1,top=NULL) 

# ,heights = c(5, 10)

3.1.4 Comparing empirical powers - with interim analysis

For each combination of (\(p_1^{(0)},p_2^{(0)},OR_1,OR_2\)), we plot the empirical power for each design -Selected Endpoint with blinded estimation (SE), Selected Endpoint with unblinded estimation (SE_u), Composite Endpoint (CE), and Relevant Endpoint (RE)- according to the correlation \(\rho\). In the empirical powers for the Selected Endpoint, we plot the points in different colors depending on which endpoint would have been chosen using the ARE calculated from the true parameters (\(p_1^{(0)},p_2^{(0)},OR_1,OR_2,\rho\)).

For each combination of (\(p_1^{(0)},p_2^{(0)},OR_1,OR_2\)), we also present a table with:

  1. Corr: The values considered for the correlation in this scenario;
  2. ARE: The value of the ARE computed using the parameters used in the design, i.e., (\(p_1^{(0)},p_2^{(0)},OR_1,OR_2\)) and \(\rho\);
  3. %CE: The percentage of cases over the 100000 replications in which the selected endpoint was the composite endpoint using blinded data;
  4. %CE_u: The percentage of cases over the 100000 replications in which the selected endpoint was the composite endpoint using unblinded data.
p <- list() 
enum = 1 
it <- 1
for(i in 1:max(dataset$scenario)){
  sub=subset(dataset,dataset$scenario==i & dataset$p_init==0.5)
  # 
  p[[enum]] <-ggplot(sub, aes(x=corr, y=Test_Power_ES, color=as.factor(decision)))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset$p0_e1[it],",",dataset$p0_e2[it],",",dataset$OR1[it],",",dataset$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (ES)", x="Correlation", color="Decision") + coord_cartesian(ylim = c(0.60, 1))+ geom_path()+ theme(plot.title = element_text(size=9),legend.position = c(0.8, 0.2))
  p[[enum+1]] <-ggplot(sub, aes(x=corr, y=Test_Power_ES_ub, color=as.factor(decision)))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset$p0_e1[it],",",dataset$p0_e2[it],",",dataset$OR1[it],",",dataset$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (ES, unblinded)", x="Correlation", color="Decision") + coord_cartesian(ylim = c(0.60, 1))+ geom_path()+ theme(plot.title = element_text(size=9),legend.position = c(0.8, 0.2))
  # 
  p[[enum+2]] <-ggplot(sub, aes(x=corr, y=Test_Power_ES_SS, color=as.factor(ss_decision)))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset$p0_e1[it],",",dataset$p0_e2[it],",",dataset$OR1[it],",",dataset$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (ES)", x="Correlation", color="Decision SS") + coord_cartesian(ylim = c(0.60, 1))+ geom_path()+ theme(plot.title = element_text(size=9),legend.position = c(0.8, 0.2))
  p[[enum+3]] <-ggplot(sub, aes(x=corr, y=Test_Power_ES_ubSS, color=as.factor(ss_decision)))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset$p0_e1[it],",",dataset$p0_e2[it],",",dataset$OR1[it],",",dataset$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (ES, unblinded)", x="Correlation", color="Decision SS") + coord_cartesian(ylim = c(0.60, 1))+ geom_path()+ theme(plot.title = element_text(size=9),legend.position = c(0.8, 0.2))
  # 
  p[[enum+4]] <-ggplot(sub, aes(x=corr, y=Test_Power_CE))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset$p0_e1[it],",",dataset$p0_e2[it],",",dataset$OR1[it],",",dataset$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (CE)", x="Correlation", color="Decision") + coord_cartesian(ylim = c(0.60, 1))+ geom_path()+ theme(plot.title = element_text(size=9))
  p[[enum+5]] <-ggplot(sub, aes(x=corr, y=Test_Power_RE))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset$p0_e1[it],",",dataset$p0_e2[it],",",dataset$OR1[it],",",dataset$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (RE)", x="Correlation", color="Decision") + coord_cartesian(ylim = c(0.60, 1))+ geom_path()+ theme(plot.title = element_text(size=9))

  stable <- data.frame(Corr=sub$corr, 
                       ARE=round(sub$ARE,2),"%CE"=round(100*sub$decision_ES,2),
                       "%CE_u"=round(100*sub$decision_ES_ub,2),check.names=FALSE)  

  p[[enum+6]] <- ggtexttable(stable, rows = NULL, theme = ttheme(base_style = "default", base_size = 9))
  
  stable <- data.frame(Corr=sub$corr, 
                       SS=round(sub$ss_ratio,2),"%CE"=round(100*sub$decision_ES_SS,2),
                       "%CE_u"=round(100*sub$decision_ES_ubSS,2),check.names=FALSE)  

  p[[enum+7]] <- ggtexttable(stable, rows = NULL, theme = ttheme(base_style = "default", base_size = 9))

  enum=enum+8 
  it <- it + dim(subset(dataset,dataset$scenario==i))
}
marrangeGrob(p,ncol=4,nrow=1,top=NULL) 

# ,heights = c(5, 10)

3.1.5 Additional scenarios - without interim

p <- list()
enum = 1
it <- 1
for(i in 1:max(dataset_SSadd$scenario)){
  sub=subset(dataset_SSadd,dataset_SSadd$scenario==i & dataset_SSadd$p_init==1.0) 
  
  p[[enum]] <-ggplot(sub, aes(x=corr, y=Test_Power_ES_SS, color=as.factor(ss_decision)))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset_SSadd$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset_SSadd$p0_e1[it],",",dataset_SSadd$p0_e2[it],",",dataset_SSadd$OR1[it],",",dataset_SSadd$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (ES)", x="Correlation", color="Decision SS") + coord_cartesian(ylim = c(0, 1))+ geom_path()+ theme(plot.title = element_text(size=9),legend.position = c(0.8, 0.2))
  p[[enum+1]] <-ggplot(sub, aes(x=corr, y=Test_Power_ES_ubSS, color=as.factor(ss_decision)))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset_SSadd$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset_SSadd$p0_e1[it],",",dataset_SSadd$p0_e2[it],",",dataset_SSadd$OR1[it],",",dataset_SSadd$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (ES, unblinded)", x="Correlation", color="Decision SS") + coord_cartesian(ylim = c(0, 1))+ geom_path()+ theme(plot.title = element_text(size=9),legend.position = c(0.8, 0.2))

  stable <- data.frame(Corr=sub$corr,
                       SS=round(sub$ss_ratio,2),"%CE"=round(100*sub$decision_ES_SS,2),
                       "%CE_u"=round(100*sub$decision_ES_ubSS,2),check.names=FALSE)

  p[[enum+2]] <- ggtexttable(stable, rows = NULL, theme = ttheme(base_style = "default", base_size = 9))

  enum=enum+3
  it <- it + dim(subset(dataset_SSadd,dataset_SSadd$scenario==i))
}
marrangeGrob(p,ncol=3,nrow=1,top=NULL)

3.1.6 Additional scenarios - with interim

p <- list()
enum = 1
it <- 1
for(i in 1:max(dataset_SSadd$scenario)){
  sub=subset(dataset_SSadd,dataset_SSadd$scenario==i & dataset_SSadd$p_init==0.5) 

  p[[enum]] <-ggplot(sub, aes(x=corr, y=Test_Power_ES_SS, color=as.factor(ss_decision)))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset_SSadd$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset_SSadd$p0_e1[it],",",dataset_SSadd$p0_e2[it],",",dataset_SSadd$OR1[it],",",dataset_SSadd$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (ES)", x="Correlation", color="Decision SS") + coord_cartesian(ylim = c(0, 1))+ geom_path()+ theme(plot.title = element_text(size=9),legend.position = c(0.8, 0.2))
  p[[enum+1]] <-ggplot(sub, aes(x=corr, y=Test_Power_ES_ubSS, color=as.factor(ss_decision)))  +
    geom_point(size=2)+ ggtitle(paste("Scenario", dataset_SSadd$scenario[it], "\n (p1,p2,OR1,OR2) \n=(", dataset_SSadd$p0_e1[it],",",dataset_SSadd$p0_e2[it],",",dataset_SSadd$OR1[it],",",dataset_SSadd$OR2[it],")"))+geom_point(size=2)  + labs(y = "Empirical Power (ES, unblinded)", x="Correlation", color="Decision SS") + coord_cartesian(ylim = c(0, 1))+ geom_path()+ theme(plot.title = element_text(size=9),legend.position = c(0.8, 0.2))

  stable <- data.frame(Corr=sub$corr,
                       SS=round(sub$ss_ratio,2),"%CE"=round(100*sub$decision_ES_SS,2),
                       "%CE_u"=round(100*sub$decision_ES_ubSS,2),check.names=FALSE)

  p[[enum+2]] <- ggtexttable(stable, rows = NULL, theme = ttheme(base_style = "default", base_size = 9))

  enum=enum+3
  it <- it + dim(subset(dataset_SSadd,dataset_SSadd$scenario==i))
}
marrangeGrob(p,ncol=3,nrow=1,top=NULL)

3.2 Under the null hypothesis (\(H_0\))

4 Decision criteria

summary(dataset)
##      p0_e1            p0_e2             OR1              OR2        
##  Min.   :0.1000   Min.   :0.1000   Min.   :0.6000   Min.   :0.7500  
##  1st Qu.:0.1000   1st Qu.:0.1000   1st Qu.:0.6000   1st Qu.:0.7500  
##  Median :0.2000   Median :0.1000   Median :0.8000   Median :0.7500  
##  Mean   :0.1512   Mean   :0.1707   Mean   :0.7025   Mean   :0.7748  
##  3rd Qu.:0.2000   3rd Qu.:0.2500   3rd Qu.:0.8000   3rd Qu.:0.8000  
##  Max.   :0.2000   Max.   :0.2500   Max.   :0.8000   Max.   :0.8000  
##     scenario          p_init         p1_e1             p1_e2        
##  Min.   : 1.000   Min.   :0.50   Min.   :0.06250   Min.   :0.07692  
##  1st Qu.: 4.000   1st Qu.:0.50   1st Qu.:0.08163   1st Qu.:0.07692  
##  Median : 9.000   Median :0.75   Median :0.13043   Median :0.08163  
##  Mean   : 8.504   Mean   :0.75   Mean   :0.11164   Mean   :0.13858  
##  3rd Qu.:13.000   3rd Qu.:1.00   3rd Qu.:0.16667   3rd Qu.:0.20000  
##  Max.   :16.000   Max.   :1.00   Max.   :0.16667   Max.   :0.21053  
##    min_corr0         min_corr1          max_corr0        max_corr1     
##  Min.   :-0.2887   Min.   :-0.23094   Min.   :0.5774   Min.   :0.5000  
##  1st Qu.:-0.2887   1st Qu.:-0.19365   1st Qu.:0.6667   1st Qu.:0.6667  
##  Median :-0.1667   Median :-0.12910   Median :0.8660   Median :0.7746  
##  Mean   :-0.1893   Mean   :-0.13939   Mean   :0.8049   Mean   :0.7771  
##  3rd Qu.:-0.1111   3rd Qu.:-0.08889   3rd Qu.:1.0000   3rd Qu.:0.8944  
##  Max.   :-0.1111   Max.   :-0.07454   Max.   :1.0000   Max.   :1.0000  
##     min_corr           max_corr           corr            p0_ce       
##  Min.   :-0.23094   Min.   :0.5000   Min.   :0.0000   Min.   :0.1180  
##  1st Qu.:-0.19365   1st Qu.:0.6667   1st Qu.:0.1000   1st Qu.:0.1810  
##  Median :-0.12910   Median :0.7746   Median :0.3000   Median :0.2614  
##  Mean   :-0.13939   Mean   :0.7635   Mean   :0.3397   Mean   :0.2522  
##  3rd Qu.:-0.08889   3rd Qu.:0.8660   3rd Qu.:0.5000   3rd Qu.:0.3120  
##  Max.   :-0.07454   Max.   :1.0000   Max.   :0.8000   Max.   :0.4000  
##      p1_ce             OR_ce             ARE         decision samplesize_e1 
##  Min.   :0.08301   Min.   :0.6321   Min.   :0.5253   CE:140   Min.   : 714  
##  1st Qu.:0.13903   1st Qu.:0.6850   1st Qu.:0.8314   RE:102   1st Qu.:1335  
##  Median :0.21055   Median :0.7522   Median :1.1630            Median :3340  
##  Mean   :0.19939   Mean   :0.7325   Mean   :1.3892            Mean   :2886  
##  3rd Qu.:0.25000   3rd Qu.:0.7842   3rd Qu.:1.7350            3rd Qu.:3340  
##  Max.   :0.34211   Max.   :0.7995   Max.   :4.0514            Max.   :6072  
##  samplesize_ce       ss_ratio      ss_decision        Test_Power_CE   
##  Min.   : 619.6   Min.   :0.5805   Length:242         Min.   :0.5953  
##  1st Qu.:1063.2   1st Qu.:0.9018   Class :character   1st Qu.:0.7657  
##  Median :1663.4   Median :1.2079   Mode  :character   Median :0.8644  
##  Mean   :1868.3   Mean   :1.4496                      Mean   :0.8533  
##  3rd Qu.:2429.6   3rd Qu.:1.7327                      3rd Qu.:0.9489  
##  Max.   :5201.4   Max.   :4.1944                      Max.   :0.9998  
##  Test_Power_RE    Test_Power_ES     decision_ES     Test_Power_ES_SS
##  Min.   :0.7981   Min.   :0.8034   Min.   :0.0000   Min.   :0.8030  
##  1st Qu.:0.8008   1st Qu.:0.8076   1st Qu.:0.0000   1st Qu.:0.8076  
##  Median :0.8020   Median :0.8665   Median :1.0000   Median :0.8656  
##  Mean   :0.8024   Mean   :0.8797   Mean   :0.5823   Mean   :0.8806  
##  3rd Qu.:0.8038   3rd Qu.:0.9497   3rd Qu.:1.0000   3rd Qu.:0.9489  
##  Max.   :0.8082   Max.   :0.9997   Max.   :1.0000   Max.   :0.9997  
##  decision_ES_SS    Test_Power_ES_ub decision_ES_ub   Test_Power_ES_ubSS
##  Min.   :0.00000   Min.   :0.8027   Min.   :0.0000   Min.   :0.8022    
##  1st Qu.:0.00364   1st Qu.:0.8079   1st Qu.:0.0000   1st Qu.:0.8092    
##  Median :1.00000   Median :0.8691   Median :1.0000   Median :0.8649    
##  Mean   :0.65606   Mean   :0.8811   Mean   :0.5833   Mean   :0.8821    
##  3rd Qu.:1.00000   3rd Qu.:0.9500   3rd Qu.:1.0000   3rd Qu.:0.9488    
##  Max.   :1.00000   Max.   :0.9997   Max.   :1.0000   Max.   :0.9997    
##  decision_ES_ubSS  Test_Reject_CE Test_Reject_RE Test_Reject_ES
##  Min.   :0.00000   Mode:logical   Mode:logical   Mode:logical  
##  1st Qu.:0.00917   NA's:242       NA's:242       NA's:242      
##  Median :1.00000                                               
##  Mean   :0.65665                                               
##  3rd Qu.:1.00000                                               
##  Max.   :1.00000                                               
##  Test_Reject_ES_ub  diff_powers       gain_power
##  Mode:logical      Min.   :-0.20778   CE:160    
##  NA's:242          1st Qu.:-0.03942   RE: 82    
##                    Median : 0.05976             
##                    Mean   : 0.05087             
##                    3rd Qu.: 0.14994             
##                    Max.   : 0.20133
dataset$diff_powers = dataset$Test_Power_CE - dataset$Test_Power_RE 
ggplot(dataset, aes(x=ARE, y=diff_powers, color=as.factor(decision))) + geom_point(size=2) + labs(y = "Difference in power (Power CE - Power RE)", x="ARE", color="Decision")

# dataset$diff_powers = dataset$Test_Power_CE - dataset$Test_Power_RE 
ggplot(dataset, aes(x=ss_ratio, y=diff_powers, color=as.factor(ss_decision))) + geom_point(size=2) + labs(y = "Difference in power (Power CE - Power RE)", x="Ratio SS", color="Decision")

5 Conclusions and future directions

Conclusions:

Future steps: